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DETAILED ACTION 



1 . This communication is responsive to Amendment A, filed 10/2/03. 

2. Claims 1-28 are pending in this application. Claims 1-2, 4-5, 1 1, 15, 17-20 and 22-25 
have been amended. This action is made Final. 

3. The text of those sections of Title 35, U.S. Code not included in this action can be found 
in a prior Office action. 



4. Claims 1-3, 6, 8 and 10-28 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Filo et al. ("Filo", US 6,215,498 Bl). 

As per claim 1, Filo teaches a system for interacting with displays and all devices that use 
such displays comprised of: 

a) a display (col. 7, line 1 and lines 40-45); 

b) a sensor or camera (col. 9, lines 30-38); 

c) a pointing device remote from the sensor or camera and that can be viewed by the 
sensor or camera, wherein the sensor or camera operates independently from the pointing device 
(col. 7, lines 19-22 and lines 51-54; col. 9, lines 50-52); 

d) a method for detecting the pointing device (col. 7, lines 19-20); and 

e) a method for establishing the mapping between the position of the pointing device and 
a corresponding location on the display (col. 7, lines 17-19). 



Claim Rejections - 35 USC § 102 
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As per claim 2, Filo teaches a system for interacting with displays and all devices that use 
such displays wherein the sensor or camera, in addition to viewing the image of the pointing 
object, can also view at least one of (i) the image of the display and (ii) the reflection or effect 
that the pointing device can produce on the display and (iii) space around the pointing device 
(col. 11, lines 32-55; col. 13, lines 34-35 and lines 10-13, 15-16; col. 9, lines 49-50). 

As per claims 3 and 8, Filo teaches a system for interacting with displays and all ( devices 
that use such displays which commands the positioning of a pointing icon on the display and 
wherein the pointing icon on the display can be registered by the sensor or camera (col. 9, line 56 
through col. 10, line 22; col. 9, lines 48-50; col. 13, lines 32-34; col. 11, lines 32-36 and lines 40- 



As per claim 6, Filo teaches a system for interacting with displays and all devices that use 
such displays wherein the pointing device is used to define a particular point or region on the 
display (col. 9, line 61 through col. 10, line 3). 

As per claim 10, Filo teaches a system for interacting with displays and all devices which 
also includes at least one of the following (col. 9, line 49 through col. 10, line 22; col. 7, lines 40- 
49; col. 8, lines 10-21, 30-37; col. 11, lines 61-63; col. 13, lines 17-24; col. 14, lines 26-32): 

a) a method for selecting or highlighting a specific item or icon on the display; 

b) a method for activating a specific process, program, or menu item represented on the 
display; and 

c) a method for writing, scribing, drawing, highlighting, annotating, or otherwise 
producing marks on the display. 



43). 
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As per claim 11, Filo teaches a method for detecting the pointing device comprising the 
steps of: 

a) retrieving of data or image from a sensor or camera, wherein the pointing device is 
remote from the sensor or camera (col. 13, lines 57-59); and 

b) analyzing the data or image from the sensor or camera to locate the pointing device in 
the data, or locating at least a set of the picture elements in the image that comprise the rendition 
of the pointing device (col. 14, lines 6-7; col. 9, lines 50-52). 

As per claim 12, Filo teaches a method for detecting the pointing device wherein 
characteristics that distinguish the pointing device from other objects in the data from the sensor 
or the image from the camera are known a priori (col. 19, lines 13-67; col. 9, lines 50-52). 

As per claims 13 and 14, Filo teaches a method for detecting the pointing device wherein 
characteristics that distinguish the pointing device from other objects in the data from the sensor 
or the image from the camera are determined based on analysis of at least one set of the data 
acquired from the sensor or one image acquired from the camera and whose rendition are present 
in the data from the sensor or in the image from the camera is obtained by acquiring at least two 
sets of data from the sensor or at least two images from the camera, one with the pointing device 
in view of the sensor or the camera and one without, and comparing the two sets with one 
another (col. 14, lines 2-17; col. 9, lines 48-56). 

As per claim 15, Filo teaches a method for detecting the pointing device wherein 
adjustments or modifications are made to the position, viewing angles, sensitivity, and other 
settings of the sensor or the camera pursuant the analysis of the data or image retrieved from the 
sensor or the camera (col. 17, lines 4-17; col. 9, lines 48-56). 



\ 
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As per claim 16, Filo teaches a method for detecting a pointing device wherein at least 
part of the procedures for the method is carried out using at least in part the computing 
mechanism available on one or more of the following: the display, or the sensor or camera, or the 
pointing device, or the device producing the signal shown on the display, or the device producing 
the pointing icon on the display (figs. 1 and 2; col. 6, lines 6-66). 

As per claim 17, Filo teaches a method for establishing a mapping between the set of 
positions that a pointing device can and assume in addressing a set of corresponding points on 
the display comprising the steps of defining the range of positions that the pointing device can 
assume, defining the boundaries of the positions that the pointing device can assume so that a 
virtual display space comprising (i) a continuous one-dimensional line, or (ii) a continuous two- 
dimensional plane, or (iii) a continuous three-dimensional volume is defined, defining the 
boundaries of the display so that a real display space comprising (i) a continuous one- 
dimensional line, or (ii) a continuous two-dimensional plane, or (iii) a continuous three- 
dimensional volume is defined, and warping the geometry of the real display space so that the 
real display space fits optimally within the boundaries of the virtual display space (figs. 7-11 and 
13; col. 19, lines 13-67; col. 3, lines 38-42; described is a method for creating a three 
dimensional world on a two dimensional display). 

As per claim 18, Filo teaches a method for establishing a mapping between the set of 
positions that a pointing device can and assume in addressing a set of corresponding points on 
the display wherein the boundaries of the set of positions that the pointing device can assume are 
obtained by querying the user to point to the boundaries (col. 12, lines 5-22). 
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As per claim 19, Filo teaches a method for establishing a mapping between the set of 
positions that a pointing device can and assume in addressing a set of corresponding points on 
the display wherein the boundaries of the set of positions that the pointing device can assume are 
obtained by contours or the periphery of the display as the display is viewed by the sensor or 
camera (col, 12, lines 5-22). 

As per claim 20, Filo teaches a method for establishing a mapping between the set of 
positions that a pointing device can and assume in addressing a set of corresponding points on 
the display wherein at least one special display image is used for establishing the mapping 
between the set of positions that the pointing device can assume in addressing a set of 
corresponding points on the display (col. 12, lines 5-22). 

As per claim 21, Filo teaches a method for establishing a mapping between the set of 
positions that a pointing device can and assume in addressing a set of corresponding points on 
the display wherein at least part of the procedures for the method is carried out using at least in 
part the computing mechanism available on one or more of the following: the display, or the 
sensor or camera, or the pointing device, or the device producing the signal shown on the 
display, or the device producing the pointing icon on the display (figs. 1 and 2; col. 6, lines 6- 
66). 

Claim 22 is similar in scope to claim 11, with the exception that the method is for 
detecting a display perspective which is inherent when there is already a method for detecting the 
pointing device in an interactive virtual world environment, and is therefore rejected under 
similar rationale. 

Claim 23 is similar in scope to claim 12 and is therefore rejected under similar rationale. 
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Claims 24 and 26 in combination is similar in scope to the combination of claims 13 and 



As per claim 25, Filo teaches a method for detecting the pointing device wherein the 
display refers to a predetermined range of positions specified by the user that the pointing device 
can assume (col. 3, lines 38-42). 

Claim 27 is similar in scope to claim 15 and is therefore rejected under similar rationale. 

Claim 28 is similar in scope to claim 16 and is therefore rejected under similar rationale. 



5. Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Filo et al. ("Filo", 
US 6,215,498 Bl) in view of Edwards et al. ("Edwards", US 6,459,442 Bl). 

As per claim 7, Filo teaches a system for interacting with displays and all devices that use 
such displays comprising a pointing device for entering information into the system and can be 
used in conjunction with various software applications (col. 7, lines 46-54; col. 8, lines 16-35; 
col. 10, lines 24-31 and line 57; col. 13, lines 21-25; information entered into the system further 
include a clipboard on which a user can draw). Filo does not explicitly disclose the pointing 
device to be used to define a vector on the plane of the display that indicates a direction and 
magnitude relative to or with respect to an item on the display or a region of the display. 
Edwards teaches a system for interacting with displays wherein the pointing device to be used to 
define a vector on the plane of the display that indicates a direction and magnitude relative to or 
with respect to an item on the display or a region of the display (col. 8, lines 12-21). Therefore, it 
would have been obvious to an artisan at the time of the invention to include Edwards' teaching 



14 and is therefore rejected under similar rationale. 



Claim Rejections - 35 USC § 103 
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of a computer aided design tool, wherein the pointing device to be used to define a vector on the 
plane of the display that indicates a direction and magnitude relative to or with respect to an item 
on the display or a region of the display, to Filo's teaching of a system for interacting with 
displays that includes drawing capabilities to provide users with an environment capable of 
incorporating multiple applications and capabilities to enhance a user's individual task. 
6. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over Filo et al. ("Filo", 
US 6,215,498 Bl) in view of Applicant's Admitted prior art. 

As per claim 9, although Filo teaches a system for interacting with displays and all 
devices that use such displays which also includes a method for sensing the pointing device's 
position relative to the position of a pointer icon on the display (col. 7, lines 1-19), Filo does not 
explicitly disclose the system to include a method for correcting the offsets between the position 
of the pointing device, or reflection, or effect thereof on the display as observed by the user or by 
the sensor or the camera, and the position of the pointer icon on the display. However, 
Applicant's admitted prior art teaches a system for interacting with displays and all devices that 
use such displays to include a method for correcting the offsets between the position of the 
pointing device, or reflection, or effect thereof on the display as observed by the user or by the 
sensor or the camera, and the position of the pointer icon on the display (page 21, lines 6-9). 
Therefore, it would have been obvious to an artisan at the time of the invention to include 
Applicant admitted prior art's teaching of a system for correcting the offsets between the position 
of the pointing device and the position of the pointer icon on the display to Filo's method of a 
system for sensing the pointing device's position relative to the position of a pointer icon on the 
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display in order to reduce the margin of error concerning the position of the pointing device and 
the position of the pointer icon on the display. 

7. Claim 4-5 are rejected under 35 U.S.C. 103(a) as being unpatentable over Filo et al. 
("Filo", US 6,215,498 Bl). 

As per claim 4, although Filo teaches a system for interacting with displays and all 
devices that use such displays wherein the pointing device is a part of the human body of a user 
such as a hand or a finger, or an ornament or device worn on the human body such as a glove or 
thimble, Filo does not explicitly disclose the sensor or camera being remote from the human 
body (figs. 2(A-B), 4, 5(A-B) and 11). Official Notice is given that sensors or camera being 
remote from the human body is well known in the art. Therefore, it would have been obvious to 
an artisan at the time of the invention to include sensors or camera being remote from the human 
body to Filo's teaching of sensors or cameras in order to provide users with an implementation 
preference. 

As per claim 5, Filo teaches a system for interacting with displays and all devices that use 
such displays wherein the pointing device is used to point to regions of the display by way of 
changing its position, attitude, or presentation (col. 9, line 61 through col. 10, line 3). 



8. Applicant's arguments with respect to claim 4 have been considered but are moot in view 
of the new ground(s) of rejection. Applicant's arguments filed in Amendment A have been fully 
considered but they are not persuasive. 
Applicant argued the following: 
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(a) Filo's movement of a pointer is not a consequence of the remote sensor viewing the 
pointing device. 

(b) Filo does not teach the camera being used to determine the location of the pointing 

device. 

(c) Filo does not teach a method for detecting a pointing device wherein the 
characteristics that distinguish the pointing device from other objects in the data from the sensor 
or the image from the camera are known at priori. 

(d) Filo does not teach detecting changes in the scene to decide whether to record the next 

frame. 

(e) Filo does not show or disclose any corrections to sensor or camera data acquisition 
parameters or viewing angles. 

(f) Edwards does not teach image acquisition from the remote camera or sensor. 

(g) Applicant's Admitted Prior Art does not show or disclose making corrections let 
alone detecting the icon or the pointing icon on the display. 

The Examiner disagrees for the following reasons: 

Per (a), in response to applicant's argument that the references fail to show certain 
features of applicant's invention, it is noted that the features upon which applicant relies (i.e., 
movement of a pointer is a consequence of the remote sensor viewing the pointing device) are 
not recited in the rejected claim(s). Although the claims are interpreted in light of the 
specification, limitations from the specification are not read into the claims. See In re Van 
Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). 
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Per (b), in response to applicant's argument that the references fail to show certain 
features of applicant's invention, it is noted that the features upon which applicant relies (i.e., the 
camera being used to determine the location of the pointing device) are not recited in the 
rejected claim(s). Although the claims are interpreted in light of the specification, limitations 
from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1 181, 26 
USPQ2d 1057 (Fed. Cir. 1993). However, the sensor or camera is able to view the pointing 
device when it is enabled and therefore is able to locate the pointing device in the data (col. 9, 
lines 50-52). 

Per (c), Filo does teach a method for detecting the pointing device wherein characteristics 
that distinguish the pointing device from other objects in the data from the sensor or the image 
from the camera are known a priori (col. 19, lines 13-67; col. 9, lines 50-52). 

Per (d), in response to applicant's argument that the references fail to show certain 
features of applicant's invention, it is noted that the features upon which applicant relies (i.e 
detecting changes in the scene to decide whether to record the next frame) are not recited in the 
rejected claim(s). Although the claims are interpreted in light of the specification, limitations 
from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1 181, 26 
USPQ2d 1057 (Fed. Cir. 1993). 

Per (e), Filo does teach a method for detecting the pointing device wherein characteristics 
that distinguish the pointing device from other objects in the data from the sensor or the image 
from the camera are determined based on analysis of at least one set of the data acquired from 
the sensor or one image acquired from the camera and whose rendition are present in the data 
from the sensor or in the image from the camera is obtained by acquiring at least two sets of data 
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from the sensor or at least two images from the camera, one with the pointing device in view of 
the sensor or the camera and one without, and comparing the two sets with one another (col. 14, 
lines 2-17; col. 9, lines 48-56; the Virtual VCR makes an initial recording of the preconditions in 
the virtual environment and then only records the changes as they occur). 

Per (f), in response to applicant's arguments against the references individually, one 
cannot show nonobviousness by attacking references individually where the rejections are based 
on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In 
re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Filo teaches image acquisition 
from the remote camera or sensor (col. 11, lines 32-55; col. 13, lines 34-35 and lines 10-13, 15- 
16; col. 9, lines 49-50). The teachings extracted from Edwards was for the pointing device to be 
used to define a vector on the plane of a display that indicates a direction and magnitude relative 
to or with respect to an item on the display or a region of the display (col 8, lines 12-21). 

Per (g), in response to applicant's arguments against the references individually, one 
cannot show nonobviousness by attacking references individually where the rejections are based 
on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In 
re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). Filo teaches detecting the icon 
or the pointing icon on the display (col. 9, lines 49-50). Applicant's Admitted Prior Art teaches 
"[a] variety of known methods, such as feedback control of the proportional (P). . .can be used for 
the correction step". 




* 
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Conclusion 



9. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 



10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Le Nguyen whose telephone number is (703) 305-7601. The 
examiner can normally be reached on Monday - Friday from 5:30 am to 2:00 pm (EST). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kristine Kincaid, can be reached on (703) 308-0640. 

The fax numbers for the organization where this application or proceeding is assigned are 
as follows: 

(703) 746-7238 [After Final Communication] 

(703) 872-9306 [Official Communication] 

(703) 746-7240 [For status inquiries, Draft Communication] 



Inquires 



* 
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Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-3900. 

Le Nguyen 
Patent Examiner 

December 22, 2003 . 





